According to the High Efficiency Video Coding (HEVC) as one of standards for moving image encoding, a prediction method is determined for each block referred to as prediction unit (PU). The block size of the PU is specified depending in each of a plurality of hierarchies in a quadtree division structure. A moving image encoding apparatus determines an optimum intra-prediction mode in each of the hierarchies of quadtree division, and compares the optimum intra-prediction modes in the hierarchies with each other to determine optimum PU division.
A moving image encoding apparatus that selects the block size based on a first evaluation value based on a sum of absolute difference values of sub blocks and a second evaluation value based on a sum of absolute values of orthogonal transformation coefficients of sub blocks after the orthogonal transformation of an original image has been known. For example, Japanese Laid-open Patent Publication No. 2010-283495 discloses related art.